Why (and When and How) Contrastive Divergence Works
نویسنده
چکیده
Contrastive divergence (CD) is a promising method of inference in high dimensional distributions with intractable normalizing constants, however, the theoretical foundations justifying its use are somewhat weak. This document proposes a framework for understanding CD inference, including how and when it works. It provides multiple justifications for the CD moment conditions, including framing them as a variational approximation. Algorithms for performing inference are discussed and are applied to social network data using an exponential-family random graph models (ERGM). The framework also provides guidance about how to construct MCMC kernels providing good CD inference, which turn out to be quite different from those used typically to provide fast global mixing.
منابع مشابه
The Why, Who, What, How, and When of Patient Engagement in Healthcare Organizations: A Response to Recent Commentaries
متن کامل
Application of qualitative research in management (why, when and how)
Application of qualitative research in management (why, when and how) Qualitative research has been increasingly used in all sciences. In this paper we will discuss about the applications of qualitative research methods in management sciences. The differences between quantitative and qualitative researches will be clarified and the statistical methods which are suitable for such researches will...
متن کاملWhy We Must Talk About Institutional Corruption to Understand Wrongdoing in the Health Sector; Comment on “We Need to Talk About Corruption in Health Systems”
While various forms of corruption are common in many health systems around the world, defining wrongdoing in terms of legality and the use of public office for private gain obstructs our understanding of its nature and intractability. To address this, I suggest, we must not only break the silence about the extent of wrongdoing in the health sector, but also talk differe...
متن کاملInformation Geometry of Contrastive Divergence
The contrastive divergence(CD) method proposed by Hinton finds an approximate solution of the maximum likelihood of complex probability models. It is known empirically that the CD method gives a high-quality estimation in a small computation time. In this paper, we give an intuitive explanation about the reason why the CD method can approximate well by using the information geometry. We further...
متن کاملWormholes Improve Contrastive Divergence
In models that define probabilities via energies, maximum likelihood learning typically involves using Markov Chain Monte Carlo to sample from the model’s distribution. If the Markov chain is started at the data distribution, learning often works well even if the chain is only run for a few time steps [3]. But if the data distribution contains modes separated by regions of very low density, bri...
متن کامل